Artificial evolution of robot bodies and control: on the interaction between evolution, individual and cultural learning

We survey and reflect on evolutionary approaches to the joint optimisation of the body and control of a robot, in scenarios where a the goal is to find a design that maximises performance on a specified task. The review is grounded in a general framework for evolution which permits the interaction of evolution acting on a population with individual and cultural learning mechanisms. We discuss examples of variations of the general scheme of "evolution plus learning" from a broad range of robotic systems, and reflect on how the interaction of the two paradigms influences diversity, performance, and rate of improvement. Finally, we suggest a number of avenues for future work as result of the insights that arise from the review

Evolution of Diverse, Manufacturable Robot Body Plans

Advances in rapid prototyping have opened up new avenues of research within Evolutionary Robotics in which not only controllers but also the body plans (morphologies) of robots can evolve in real-time and real-space. However, this also introduces new challenges, in that robot models that can be instantiated from an encoding in simulation might not be manufacturable in practice (due to constraints associated with the 3D printing and/or automated assembly processes). We introduce a representation for evolving (wheeled) robots with a printed plastic skeleton, and evaluate three variants of a novelty-search algorithm in terms of their ability to produce populations of manufacturable but diverse robots. While the set of manufacturable robots discovered represent only a small fraction of the overall search space of all robots, all methods are shown to be capable of generating a diverse population of manufacturable robots that we conjecture is large enough to seed an evolving robotic ecosystem

Hardware Design for Autonomous Robot Evolution

The long term goal of the Autonomous Robot Evolution (ARE) project is to create populations of physical robots, in which both the controllers and body plans are evolved. The transition for evolutionary designs from purely simulation environments into the real world creates the possibility for new types of system able to adapt to unknown and changing environments. In this paper, a system for creating robots is introduced in order to allow for their body plans to be designed algorithmically and physically instantiated using the previously introduced Robot Fabricator. This system consists of two types of components. Firstly, \textit{skeleton} parts are created bespoke for each design by 3D printing, allowing the overall shape of the robot to include almost infinite variety. To allow for the shortcomings of 3D printing, the second type of component are \textit{organs} which contain components such as motors and sensors, and can be attached to the skeleton to provide particular functions. Specific organ designs are presented, with discussion of the design challenges for evolutionary robotics in hardware. The Robot Fabricator is extended to allow for robots with joints, and some example body plans shown to demonstrate the diversity possible using this system of robot generation

DREAM Architecture: a Developmental Approach to Open-Ended Learning in Robotics

Robots are still limited to controlled conditions, that the robot designer knows with enough details to endow the robot with the appropriate models or behaviors. Learning algorithms add some flexibility with the ability to discover the appropriate behavior given either some demonstrations or a reward to guide its exploration with a reinforcement learning algorithm. Reinforcement learning algorithms rely on the definition of state and action spaces that define reachable behaviors. Their adaptation capability critically depends on the representations of these spaces: small and discrete spaces result in fast learning while large and continuous spaces are challenging and either require a long training period or prevent the robot from converging to an appropriate behavior. Beside the operational cycle of policy execution and the learning cycle, which works at a slower time scale to acquire new policies, we introduce the redescription cycle, a third cycle working at an even slower time scale to generate or adapt the required representations to the robot, its environment and the task. We introduce the challenges raised by this cycle and we present DREAM (Deferred Restructuring of Experience in Autonomous Machines), a developmental cognitive architecture to bootstrap this redescription process stage by stage, build new state representations with appropriate motivations, and transfer the acquired knowledge across domains or tasks or even across robots. We describe results obtained so far with this approach and end up with a discussion of the questions it raises in Neuroscience

Practical Hardware for Evolvable Robots

The evolutionary robotics field offers the possibility of autonomously generating robots that are adapted to desired tasks by iteratively optimising across successive generations of robots with varying configurations until a high-performing candidate is found. The prohibitive time and cost of actually building this many robots means that most evolutionary robotics work is conducted in simulation, but to apply evolved robots to real-world problems, they must be implemented in hardware, which brings new challenges. This paper explores in detail the design of an example system for realising diverse evolved robot bodies, and specifically how this interacts with the evolutionary process. We discover that every aspect of the hardware implementation introduces constraints that change the evolutionary space, and exploring this interplay between hardware constraints and evolution is the key contribution of this paper. In simulation, any robot that can be defined by a suitable genetic representation can be implemented and evaluated, but in hardware, real-world limitations like manufacturing/assembly constraints and electrical power delivery mean that many of these robots cannot be built, or will malfunction in operation. This presents the novel challenge of how to constrain an evolutionary process within the space of evolvable phenotypes to only those regions that are practically feasible: the viable phenotype space. Methods of phenotype filtering and repair were introduced to address this, and found to degrade the diversity of the robot population and impede traversal of the exploration space. Furthermore, the degrees of freedom permitted by the hardware constraints were found to be poorly matched to the types of morphological variation that would be the most useful in the target environment. Consequently, the ability of the evolutionary process to generate robots with effective adaptations was greatly reduced. The conclusions from this are twofold. 1) Designing a hardware platform for evolving robots requires different thinking, in which all design decisions should be made with reference to their impact on the viable phenotype space. 2) It is insufficient to just evolve robots in simulation without detailed consideration of how they will be implemented in hardware, because the hardware constraints have a profound impact on the evolutionary space

Practical hardware for evolvable robots

The evolutionary robotics field offers the possibility of autonomously generating robots that are adapted to desired tasks by iteratively optimising across successive generations of robots with varying configurations until a high-performing candidate is found. The prohibitive time and cost of actually building this many robots means that most evolutionary robotics work is conducted in simulation, but to apply evolved robots to real-world problems, they must be implemented in hardware, which brings new challenges. This paper explores in detail the design of an example system for realising diverse evolved robot bodies, and specifically how this interacts with the evolutionary process. We discover that every aspect of the hardware implementation introduces constraints that change the evolutionary space, and exploring this interplay between hardware constraints and evolution is the key contribution of this paper. In simulation, any robot that can be defined by a suitable genetic representation can be implemented and evaluated, but in hardware, real-world limitations like manufacturing/assembly constraints and electrical power delivery mean that many of these robots cannot be built, or will malfunction in operation. This presents the novel challenge of how to constrain an evolutionary process within the space of evolvable phenotypes to only those regions that are practically feasible: the viable phenotype space. Methods of phenotype filtering and repair were introduced to address this, and found to degrade the diversity of the robot population and impede traversal of the exploration space. Furthermore, the degrees of freedom permitted by the hardware constraints were found to be poorly matched to the types of morphological variation that would be the most useful in the target environment. Consequently, the ability of the evolutionary process to generate robots with effective adaptations was greatly reduced. The conclusions from this are twofold. 1) Designing a hardware platform for evolving robots requires different thinking, in which all design decisions should be made with reference to their impact on the viable phenotype space. 2) It is insufficient to just evolve robots in simulation without detailed consideration of how they will be implemented in hardware, because the hardware constraints have a profound impact on the evolutionary space

Hardware Design for Autonomous Robot Evolution

The long term goal of the Autonomous Robot Evolution (ARE) project is to create populations of physical robots, in which both the controllers and body plans are evolved. The transition of evolutionary designs from purely simulation environments into the real world creates the possibility for new types of system able to adapt to unknown and changing environments. In this paper, a system for creating robots is introduced in order to allow for their body plans to be designed algorithmically and physically instantiated using the previously introduced Robot Fabricator. This system consists of two types of components. Firstly, skeleton parts are created bespoke for each design by 3D printing, allowing the overall shape of the robot to include almost infinite variety. To allow for the shortcomings of 3D printing, the second type of component are organs which contain components such as motors and sensors, and can be attached to the skeleton to provide particular functions. Specific organ designs are presented, with discussion of the design challenges for evolutionary robotics in hardware. The Robot Fabricator is extended to allow for robots with joints, and some example body plans shown to demonstrate the diversity possible using this system of robot generation

Evaluation of frameworks that combine evolution and learning to design robots in complex morphological spaces

Jointly optimising both the body and brain of a robot is known to be a challenging task, especially when attempting to evolve designs in simulation that will subsequently be built in the real world. To address this, it is increasingly common to combine evolution with a learning algorithm that can either improve the inherited controllers of new offspring to fine tune them to the new body design or learn them from scratch. In this paper an approach is proposed in which a robot is specified indirectly by two compositional pattern producing networks (CPPN) encoded in a single genome, one which encodes the brain and the other the body. The body part of the genome is evolved using an evolutionary algorithm (EA), with an individual learning algorithm (also an EA) applied to the inherited controller to improve it. The goal of this paper is to determine how to utilise the results of learning process most effectively to improve task performance of the robot. Specifically, three variants are investigated: (1) evolution of the body+controller only; (2) a learning algorithm is applied to the inherited controller with the learned fitness assigned to the genome; (3) learning is applied and the genome is updated with the learned controller, as well as being assigned the learned fitness. Experiments are performed in three different scenarios chosen to favour different bodies and locomotion patterns. It is shown that better performance can be obtained using learning but only if the learned controller is inherited by the offspring